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ABSTRACT:  A  strong  need  exists  for  a  simulation-based  means  to  provide  individuals  in  deployed  settings 
teamwork  and  cross-platform  coordination  skills  training  in  realistic,  mission-oriented  scenarios.  This  need  can  be 
met  by  using  advanced  human  behavioral  representation  technology  to  provide  synthetic  teammates  and  collateral 
entities  operating  within  a  sophisticated  synthetic  battlespace.  When  combined  with  a  full  range  of  automated, 
intelligent  agent-based  instructional  support  capabilities  including  real-time  performance  measurement,  diagnosis 
and  feedback  along  with  menu-driven  scenario  generation  and  replay  capability  for  debriefing  purposes  the  result  is 
a  system  called  Synthetic  Cognition  for  Operational  Team  Training  (SCOTT).  An  initial  SCOTT  system  is  being 
developed  to  provide  training  in  cross-platform  coordination  skills  for  members  of  the  Navy  E-2C  Hawkeye  tactical 
crew.  The  architecture  and  behavioral  representation  issues  in  E-2C  SCOTT  are  discussed. 


1.  Introduction 

A  common  requirement  in  naval  aviation  operations  is  for 
a  team  of  operators  to  work  together  efficiently  and 
effectively  to  attain  coordinated  mission  success. 
Unfortunately,  insufficient  resources  are  available  to 
ensure  that  aviators  receive  the  practice  and  feedback 
required  for  them  to  maintain  their  team  skills  in 
deployed  environments.  The  cause  is  twofold:  first, 
technologies  and  methodologies  needed  for  deployed  team 
training  are  inadequate.  Second,  such  training  resources 
as  instructors,  ranges,  and  ordnance  are  lacking, 
particularly  in  the  deployed  environment.  Therefore,  the 
development  and  demonstration  of  a  theoretically  sound 
means  to  training  advanced  team  skills — one  that  will 
allow  aviators  to  practice  in  the  absence  of  trained 
instructors  or  team  members— is  crucially  needed. 

The  Synthetic  Cognition  for  Operational  Team  Training 
(SCOTT)  project,  sponsored  by  the  Office  of  Naval 
Research,  is  a  response  to  this  need.  SCOTT  is  an  effort 
to  apply  advanced  human  behavioral  representation 
methods  to  meet  a  critical  team  training  need  in  the  Navy, 
specifically  to  provide  deployable  training  for  critical 
aviation  team  skills  without  requiring  any  additional 
personnel  besides  the  individual  receiving  the  training. 


SCOTT  will  allow  aviators  to  practice  in  the  absence  of 
trained  instructors  or  live  teammates  through  the  use  of: 

•  synthetic  teammates  and  adversaries  in  a  synthetic 
battlespace, 

•  synthetic  instructors  to  provide  performance 
assessment,  measurement,  feedback,  and  training 
management. 

The  synthetic  teammates  pose  a  unique  challenge  in 
SCOTT  because  they  must  engage  the  trainee  in  realistic 
teamwork  behaviors  in  a  believable  manner.  Not  only 
does  this  require  them  to  carry  out  reasoning  needed  to 
generate  collaborative  and  teamwork  behaviors  but,  in  the 
Naval  aviation  context,  it  also  requires  them  to  carry  out 
the  majority  of  these  behaviors  verbally. 

The  key  issues  being  addressed  in  SCOTT  are  detailed 
below.  In  Section  2,  the  training  issues  involved  in 
cross-platform  aviation  team  training  are  discussed  along 
with  a  brief  description  of  the  operational  context  for 
which  SCOTT  is  being  developed.  Next,  the  SCOTT 
approach  to  developing  synthetic  teammates  is  presented, 
with  particular  attention  to  the  issues  of  developing  a 
synthetic  teammate  that  can  provide  both  teamwork 
practice  opportunities  and  tutoring/instruction.  Finally, 
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the  SCOTT  architecture  and  component  details  are 
presented . 

2.  The  Aviation  Cross-Platform  Team 
Training  Problem 

Currently,  fleet  aviation  units  lack  advanced  team  skills 
training  opportunities,  although  it  has  been  widely 
acknowledged  that  advanced  team  skills  are  crucial  to  the 
safe  and  effective  operation  of  Naval  aviation  assets.  For 
example,  poor  team  skills  contributed  to  more  than  80% 
of  370  Navy/Marine  Corps  aviation  mishaps  that  were 
attributable  to  human  error  [1].  Hartel  et  al.  reported 
situation  awareness  problems  contributed  to  60%  of  the 
incidents,  and  decision  making  errors  contributed  to  50% 
of  the  incidents.  As  a  result,  the  Navy  research 
community’s  interest  has  increased  for  developing 
advanced  training  methods,  tools,  and  strategies  to 
enhance  team  skills  [2]. 

Recently,  researchers  have  identified  two  significant 
challenges  associated  with  team  training: 

a)  team  skills  training  needs  to  be  mission  and  context 
specific  [3];  and 

b)  complex  cognitive  skills  and  team  process  behaviors 
are  highly  perishable;  therefore,  frequent  refresher 
training  on  advanced  team  and  decision  making  skills 
is  required  [3], 

Such  training  challenges  are  of  particular  relevance  for 
deployed  aviation  crews.  It  is  likely  that  team  skills 
required  to  perform  a  given  operational  mission  are 
deficient  due  to  lack  of  recent  practice,  or  lack  of  practice 
under  mission  specific  parameters.  Training  is  needed  in 
deployed  environments  to  provide  aircrews  with  an 
opportunity  to  refresh  team  skills  related  to  the  type  of 
mission  that  the  crew  is  about  to  perform,  the  type  of 
enemy  about  to  be  encountered,  and  the  type  of  terrain  to 
be  involved. 

The  need  for  effective,  comprehensive  organic  training 
becomes  increasingly  crucial  as  the  Navy  moves  toward 
operations  in  littoral  contexts  where  practicing  skills  in  an 
aircraft  are  impossible  due  to  political  and  security  issues. 
Furthermore,  fleet  personnel  do  not  have  access  to  the 
necessary  training  resources  while  deployed.  Shore-based 
training  technologies  and  strategies  are  both  space  and 
personnel  prohibitive  for  shipboard  and  forward  deployed 
units.  Moreover,  while  the  need  for  practice-based 
teamtraining  increases,  the  realities  of  optimized  manning 
suggest  that  there  will  be  fewer  deployed  training 
supervisors  in  the  future,  as  well  as  few  opportunities  to 
engage  multiple  platforms  and  personnel  in  live  exercises 
requiring  intelligent  tutoring  training  technology.  The 
need  to  control  costs  will  further  limit  the  opportunity  for 
live  exercise-based  team  training.  The  average  annual 
training  cost  per  student  across  the  services  exceeded 
$72,500  in  FY95  [4].  Additionally,  the  costs  associated 
with  live  exercises  and  training  events  (such  as  Fleet 


Exercises  and  Air  Wing  Detachments  to  NSWC)  involves 
millions  of  dollars  across  the  turn  around  training  cycle. 

Thus,  a  way  had  to  be  found  that  provides  team  training, 
particularly  in  areas  of  cross-platform  coordination,  while 
using  minimal  manpower  and  making  minimal  demands 
on  such  operational  resources  as  fuel,  ranges,  and 
ordnance.  The  SCOTT  solution  entails: 

•  synthetic  battlespaces  (eliminating  the  need  for  use  of 
live  systems)  based  on  constructive  simulations,  in 
which  a  single  human  trainee  can  interact  with 

•  synthetic  teammates,  based  on  cognitive  modeling  and 
human  behavioral  representation  technology,  and  in 
which  the  instruction  is  provided  by 

•  intelligent  tutoring  system  technologies,  including 
use  of  instructional  agents  for  real-time  assessment, 
diagnosis,  and  feedback. 

While  the  synthetic  battlespace  technology  has  rapidly 
matured  in  recent  years,  the  latter  two  components  of 
SCOTT  represent  major  challenges.  The  technology  for 
creating  believable  synthetic  human  behavioral 
representations  is  still  emerging,  and  the  aviation 
teamwork  environment,  as  discussed  below,  poses  a 
challenging  case.  In  addition,  while  intelligent  tutoring 
has  matured  rapidly  in  recent  years  its  primary  focus  has 
been  on  individual-  skills  and  knowledge  training,  and 
based  on  direct  interactions  between  the  trainee  and  the 
simulation  environment.  The  expansion  to  team  training, 
in  which  interactions  among  human  team  members  are 
part  of  the  diagnostic  space,  as  well  as  human-computer 
interactions,  represents  a  significant  expansion  of  the 
existing  state  of  the  art. 

2.1  The  Operational  Context 

Before  discussing  the  technology  issues,  it  is  useful  to 
briefly  review  the  operational  context  for  which  SCOTT 
is  being  developed.  The  program  is  focusing  on  cross¬ 
platform  coordination  for  carrier-based  naval  aviation, 
primarily  on  the  E-2C  Elawkeye  aircraft.  Analogous  to 
the  Air  Force  E-3  AWACS,  the  Hawkeye  provides  a 
variety  of  airborne  command,  control,  communications 
and  intelligence  functions  for  Naval  missions.  The 
tactical  crew  of  the  E-2C  consists  of  a  CICO  (Combat 
Information  Center  Officer),  ACO  (Air  Control  Officer), 
and  RO  (Radar  Officer),  in  addition  to  the  pilot  and  co¬ 
pilot.  The  tactical  crew  must  interact  with  other  airborne 
platforms,  including  tactical  aircraft  (e.g.,  FA-18,  F-14), 
electronics  aircraft  (e.g.,  EA-6B,EP-3),  tankers  (e.g.,  KC- 
135),  and  other  command  and  control  aircraft  (e.g., 
AWACS),  as  well  as  surface  platforms  involved  in  air 
space  management  (e.g.,  the  Combat  Information  Center- 
CIC  on  Aegis  ships  in  the  battlegroup).  Ideally,  SCOTT 
would  eventually  provide  tools  for  developing  synthetic 
teammates  for  all  on-platform  team  members  (ACO, 
CICO,  RO),  as  well  as  off-platform  friendly  airborne  and 
surface  entities.  This  would  allow  any  member  of  the  E- 
2C  aircrew  to  train  in  working  across  this  complete  space 
of  coordinating  platforms. 


In  the  initial  development  cycle  reported  here,  the  focus  is 
explicitly  on  providing  a  training  capability  for  the  ACO. 
The  primary  synthetic  teammate  or  synther  [5]  being 
implemented  in  this  development  cycle  is  a  CICO.  The 
CICO  synther  must  be  able  to  perform  the  command  and 
control  tasks  of  the  CICO  in  the  context  of  strike  warfare 
mission  scenarios.  These  functions  are  complex, 
including  perceptual  processing  of  displayed  sensor  data, 
situation  assessment,  reasoning  and  decision  making  for 
appropriate  courses  of  action,  determining  the  information 
requirements  of  other  entities  engaged  in  the  mission, 
executing  control  actions,  and  communicating 
information  to  teammates  at  appropriate  times.  These 
issues  are  discussed  in  more  detail  below. 

An  initial  scenario  and  demonstration  will  feature  a 
constrained  battlespace  to  include  E-2C  tactical  crew 
interactions  with  various  aircraft  in  a  strike  mission.  Of 
particular  concern  is  including  intelligent  tactical  aircraft 
(e.g.,  F-18,  F-14)  simulations  that  will  allow  the  ACO 
trainee  and  the  CICO  synther  to  interact  in  complex  and 
non-scripted  interactions  in  the  battlespace.  The  specific 
scenario  being  used  is  a  large  scale,  medium  range,  strike 
mission  conducted  by  carrier  airwing  assets  that  are 
controlled  by  a  Navy  E-2C  Hawkeye  crew.  This  strike  is 
executed  in  conjunction  with  joint  assets  against  a  well 
defended  target  located  deep  inside  a  fictitious  county. 

The  joint  strike  package  involved  in  this  scenario  is 
divided  into  four  categories  based  on  mission  function: 
Sweep,  Attached  TARCAP,  Strikers  and  Support  Units. 
The  Sweep  units  are  the  fighter  aircraft  that  fly  out  ahead 
of  the  strike  package  addressing  airborne  threats  through 
the  ingress  route,  target  area  and  egress  route.  The 
Attached  TARCAP  units  are  fighter  aircraft  assigned  to 
provide  close  escort  for  the  strike  aircraft.  The  Striker 
units  are  the  aircraft  that  are  tasked  with  delivering  bombs 
on  target.  The  Support  units  are  the  aircraft  that  perfonn 
all  of  the  required  support  duties  for  the  main  strike 
package  units.  Specific  support  functions  include  strike 
and  fighter  control,  suppression  of  enemy  air  defense 
(SEAD),  electronic  surveillance,  and  various  intelligence 
units,  delousing  of  returning  aircraft  and  tanking. 

In  addition  to  controlling  the  large  scale  strike  mission, 
the  E-2C  crew  is  also  required  to  check-in  and  coordinate 
the  handoff  of  close  air  support  (CAS)  assets  to  be 
employed  in  the  Marine  Amphibious  Operating  Area 
(AOA).  Individual  E-2C  crewmember  mission 
responsibilities  are  designed  to  most  efficiently  support 
the  various  strike  elements  and  warfare  commander  needs. 
The  ACO  is  assigned  the  fighter  control  duties  in  support 
of  the  Sweep  package.  The  ACO  communicates  directly 
with  the  Air  Warfare  Commander  (AW)  as  required. 
Additionally,  the  ACO  is  the  primary  manager  of  the 
Hawkeye’s  electronic  surveillance  (ES)  equipment.  The 
Radar  Officer  (RO)  is  assigned  the  duties  of  fighter 
control  for  the  Attached  TARCAP  assets  in  addition  to 
general  situation  awareness  (SA)  calls  for  strike  aircraft. 
The  RO  ensures  optimum  perfonnance  of  the  E-2C 
weapon  system.  In  addition  to  ensuring  proper  data  link 


operation,  the  CICO  oversees  all  aspects  of  the  strike 
mission  to  ensure  complete  mission  success.  The  CICO 
coordinates  with  external  units  for  most  strike  support 
issues  and  will  ensure  that  the  various  warfare 
commanders  are  kept  appraised  of  the  current  tactical 
situation  as  required.  Virtually  all  of  these  activities  are 
carried  out  through  voice  communications  (both  within 
the  E-2C  and  with  other  aircraft),  and  interactions  with 
the  E-2C  crewstations. 

3.  SCOTT  System  Architecture 

The  SCOTT  system  is  designed  to  be  part  of  an 
intelligent  guided  practice  process  [6],  in  which  practice- 
based  training  is  only  one  part  of  a  larger  continuous 
training  cycle.  Each  iteration  of  the  cycle  prepares  for, 
conducts,  and  analyzes  a  single  simulated  problem¬ 
solving  scenario,  in  which  the  trainee  interacts  with  a 
realistic  environment.  The  cycle  begins  with  the 
specification  of  training  objectives  that  are  selected  from 
pre-defined  inventories  of  job  knowledge  and  skills  and  a 
database  on  the  individual's  past  performance  and 
training.  These  objectives  are  then  used  to  select  (or  if 
necessary,  generate)  a  problem  scenario  that  will  require 
the  use  of  the  targeted  knowledge,  skills,  and  tasks.  As 
the  problem  scenario  is  run,  pre-defined  performance 
measures  are  applied  to  trainee  actions,  and  used  to  drive 
the  diagnosis  of  trainee  performance  in  terms  of  the 
stated  objectives.  This  diagnosis  process  drives  both  the 
generation  of  real-(and  near-real-)  time  instruction  and 
feedback  to  the  trainee,  and  assessment  of  trainee 
improvement  and  needs  for  future  training.  The  training 
architecture  is  divided  into  three  parts.  The  main  part  is 
the  practice  scenario  execution,  in  which  the  trainee 
performs  a  simulated  mission  in  a  synthetic  battlespace. 
The  second  part  is  the  training  management  component, 
which  sets  up  and  initializes  the  problem  scenario  based 
on  the  individual’s  training  needs.  Once  the  simulated 
mission  is  completed,  the  third  part  comes  into  play. 
This  is  the  performance  assessment  and  After  Action 
Review  (AAR)  component,  which  performs  a  detailed 
analysis  of  trainee  performance  during  the  scenario  (using 
the  predefined  measures)  and  provides  automated 
assessment,  instructional  feedback,  and  builds  a  detailed 
AAR  report. 

The  overall  architecture  of  the  Version  1  SCOTT  System 
is  shown  in  Figure  1.  The  practice  scenario  execution 
portion  of  the  system  is  implemented  as  an  HLA 
federation,  consisting  of  a  battlespace  simulation  based  on 
JSAF  and  TacAir  Soar,  and  an  E-2C  federate  being 
developed  under  SCOTT.  This  organization  allows 
additional  federates  to  be  added  later  (e.g.,  an  AEGIS 
ship).  The  training  management  component  is  shown  on 
the  upper  left  of  Figure  1,  and  the  performance  assessment 
component  in  the  lower  left.  Each  is  briefly  discussed 
below. 


BATTLESPACE  Federate 


Figure  1.  SCOTT  Architecture 


3.1  Battlespace  Federates 

The  battlespace  federates  simulate  a  synthetic  theater  of 
war  using  the  Joint  SAF  (JSAF)  simulation  software  [7]. 
The  E-2C  federate,  which  simulates  the  aircraft  and  its 
pilot,  is  derived  from  an  E-2C  JSAF  entity,  augmented 
with  a  simulation  of  the  internal  crewstations  and  crew. 
Other  aircraft  in  the  scenario  are  also  simulated  as  JSAF 
entities,  primarily  those  that  are  not  involved  in  direct 
interactions  with  the  E-2C  tactical  crew.  Those  vehicles 
that  do  interact  directly  with  the  E-2C  tactical  crew  are 
simulated  as  intelligent  air  platforms,  based  on  the  TacAir 
Soar  technology  [8].  The  intelligent  air  platforms 
communicate  with  other  elements  within  JSAF  using 
standard  JSAF  communications  protocols,  with  one 
exception  -  voice  communications  with  human  trainee(s) 
and  synthetic  teammates  in  the  E-2C  federate.  These 
communications  are  not  supported  by  either  JSAF  or 
HLA  protocols,  and  are  therefore  handled  through  a 
separate  voice  communication  network.  The  network 
supports  three  functions: 

1 )  recognition  of  speech  from  a  human  trainee  in  the  E- 
2C  (federate)  and  translation  to  a  digital  representation 
understandable  to  the  TacAir  Soar  synthetic  pilot; 

2)  generation  of  synthetic  speech  from  a  TacAir  Soar 
synthetic  pilot  over  the  voice  network;  and 

3)  communication  of  text-based  message  content 
between  TacAir  Soar  synthetic  pilots,  and  iGEN-based 
synthetic  teammates  onboard  the  E-2C. 

See  [9]  for  additional  details  on  the  issues  involved  in 
voice  interactions  among  human  and  synthetic  entities  in 
a  distributed  constructive  simulation. 

3.2  E-2C  Federate 

The  E-2C  federate  represents  the  tactical  crew  of  the  E-2C 
aircraft  within  the  synthetic  battlespace.  Within  this 
federate,  each  tactical  crewmember  (ACO,  CICO,  RO) 
may  be  either  a  human  trainee  or  a  synthetic  teammate. 
Thus,  each  must  interact  with  the  battlespace  in  the  same 
manner  as  a  human  Naval  Flight  Officer  (NFO)  in  a  real 
E-2C,  i.e.,  through  the  tactical  crewstation  on-board  the 
aircraft.  In  SCOTT,  these  tactical  crewstations  are 
emulated  on  desktop  (i.e.,  Windows-based)  machines, 
using  a  rehosted  version  of  an  E-2C  simulator  called  E- 
TRACS  [10], 

The  current  version  of  SCOTT  contains  a  full  synther  for 
the  CICO  position.  The  RO  is  a  rudimentary  synther  , 
acting  primarily  as  a  role-player.  In  order  for  each  of 
these  two  synthers  to  interact  with  the  emulated 
crewstations,  a  software  ‘adapter’  is  provided.  This 
adapter  allows  the  synther  access  to  all  display  contents 
and  display  dynamics,  and  allows  the  synther  to 
manipulate  all  controls  at  the  crewstation. 

The  information  on  the  crewstation  displays  (particularly 
the  tactical  plots)  are  generated  through  an  E-TRACS 
component  that  simulates  the  processing  done  by  the  E- 


2C  avionics  system.  It  uses,  as  inputs,  data  on  the 
battlespace  that  represents  the  data  that  the  E-2C  would 
receive  from  its  sensors  and  data  links  (albeit  at  an 
unclassified  level).  This  data  is  obtained,  as  needed,  from 
the  battlespace  federates  using  HLA-based 
communications.  Most  interactions  that  the 
trainee/synthers  have  with  the  crewstations  simply  affect 
the  way  information  is  displayed/processed  by  the 
avionics.  The  crewstation  interactions  do  not  affect  the 
battlespace.  Rather,  the  command  and  control  functions 
of  the  E-2C  tactical  crew  are  conducted  via  voice 
channels.  As  noted  above,  this  is  done  using  a  separate 
voice  communication  network  that  spans  the  battlespace 
and  E-2C  federates. 

The  voice  communication  system  uses  two  separate 
channels.  One  channel  manages  communications  within 
the  E-2C  tactical  crew,  and  the  other  channel  manages 
communications  between  the  E-2C  tactical  crew  and 
external  platforms.  This  was  primarily  an  engineering 
decision,  based  on  the  fact  that  that  external  TacAir  Soar 
intelligent  entities  used  a  different  voice  recognition 
framework  than  that  needed  by  the  on-platform  synthers. 
It  is  important  to  recognize  that  both  on-platform  and  off- 
platform  communications  are  not  just  “point-to-point”. 
Communications  by  any  E-2C  crewmember  can  be 
overheard  by  any  teammate.  This  is  important  to  the 
teamwork  functions  on-board  the  aircraft  and  to  the 
coordination  between  the  E-2C  and  other  friendly 
platforms,  and  thus  to  the  design  and  functioning  of  the 
synthers. 

The  final  component  of  the  E-2C  federate  is  an  intelligent 
instructional  agent  software  component  that  observes, 
assesses,  and  critiques  the  activities  of  the  ACO  trainee 
within  the  current  context  of  the  (simulated)  mission.  In 
some  cases,  it  provides  feedback  directly  to  the  trainee, 
(primarily  via  the  voice  channel),  and  other  times  it 
provides  its  assessment  to  the  post-exercise  assessment 
component  of  the  system.  The  synthetic  teammates  and 
the  instructional  agent  are  discussed  in  more  detail  in 
Section  4  below. 

3.3  Training  Management  Components 

Although  behavioral  representation  simulation  issues  are 
critical  in  SCOTT,  the  overall  goal  of  the  system  is  to 
provide  a  means  of  training.  Training  management 
within  SCOTT  is  based  on  the  intelligent  guided  practice 
training  model  discussed  above.  The  main  functions  of 
the  training  management  system  are  to: 

•  track  trainee  performance  and  characteristics  across 
guided  practice  scenarios,  and 

•  use  that  perfonnance  history  to  generate  new  practice 
scenarios. 

In  the  scenario  generation  process,  the  new  scenario 
characteristics  are  based  on  training  requirements  and 
objectives  for  the  position  and  on  the  trainee’s  prior 
practice  results  against  those  objectives.  Thus,  as  a  new 


practice  scenario  is  created  it  is  associated  with  specific 
training  objectives  to  be  pursued  during  its  execution. 
This,  in  turn,  activates  objective-specific  performance 
measurement  requirements  which  are  passed  to  the 
performance  measurements  components,  thus  focusing 
trainee  assessments  on  the  specific  objectives  of  the  just- 
created  training  scenario.  The  technology  used  to 
implement  this  training  management  functionality  is 
adapted  from  the  training  management  system  described 
in  [15]. 

3.4  Performance  Assessment  Components 

A  key  difference  between  a  simple  simulation  system  and 
a  training  system  is  the  presence  of  performance 
assessment  and  feedback  in  the  latter.  SCOTT  provides 
performance  assessment  via  two  different  channels: 

•  from  the  instructional  agent  or  synthetic  instructor 
components  of  the  synthers  within  the  E-2C  federate, 
and 

•  from  a  post-processing  analysis  of  trainee  behavior. 

The  instructional  agent  (discussed  below)  is  an  extension 
of  the  CICO  synther  that  uses  the  synther’s  domain  and 
task  knowledge  to  provide  an  on-going  stream  of  context- 
based  observations  and  assessments  of  trainee  behavior. 
These  assessments  may  be  diagnostic  at  either  the 
behavioral  level  (e.g.,  trainee  failed  to  provide  timely 
calls  to  track  ‘X’)  or  at  the  cognitive  level  (e.g.,  trainee 
does  not  understand  procedures  for  ’X’),  or  simply 
context-based  events.  While  all  of  the  observations  and 
assessments  made  by  the  instructional  agent  are  stored  for 
post-run  analysis,  some  are  used  to  generate  real-time 
feedback  to  the  trainee.  In  most  cases,  the  real-time 
feedback  is  structured  so  as  to  be  delivered  by  the  CICO 
synther  as  a  form  of  ‘coaching’  to  the  ACO  trainee. 

SCOTT  also  maintains  a  comprehensive  data  collection 
process  to  create  a  body  of  data  for  performance 
measurement  post-processing.  In  addition  to  instructional 
agent  outputs,  this  data  repository  includes  all  data  on  the 
behaviors  of  the  entities  in  the  synthetic  battlespace  (via 
the  standard  JSAF  logging  facility),  plus  all  voice 
communications  (reduced  to  textual  form,  as  recognized 
or  generated  by  the  system  software)  internal  to  the  E-2C 
and  between  the  E-2C  and  other  entities,  plus  all 
keyboard  actions  and  display  contents  of  the  human  ACO 
trainee.  From  this  rich  repository,  the  post-scenario 
performance  assessment  is  conducted.  This  analysis  is 
more  exhaustive  than  that  conducted  by  the  instructional 
agent  during  the  scenario,  and  it  is  able  to  engage  in 
multi-pass  analysis  in  a  way  that  the  real-time 
instructional  agent  can  not.  The  assessment  process  is 
structured  by  data  collection  inputs  and  standards  for 
measurement/assessment  provided  by  the  training 
management  components  based  on  the  objectives 
involved.  The  assessment  process  is  designed  to  be  able 
to  accept  highly  structured  inputs  from  a  human 
observer/instructor  using  a  specialized  interface,  but  this 
is  not  necessary  nor  is  this  considered  the  normal  mode  of 


use  of  the  system  which  is  intended  to  be  without  a  live 
instructor).  The  performance  assessment  process  produces 
inputs  to  the  after  action  review  which  is  provided  to  the 
trainee  after  the  analysis  is  completed. 

4.  Synthetic  Teammates  and  Instructors 

One  of  the  primary  challenges  in  SCOTT  was  the  need  to 
provide  human  behavioral  representations  -  synthers  —  for 
controlling  a  number  of  on  and  off-platform  battlespace 
entities,  as  shown  in  Figure  2.  These  two  types  of 
entities  required  different  engineering  solutions.  The  off- 
platform  entities,  such  as  the  F-14  sweep  package,  interact 
with  the  ACO  trainee  in  the  same  way  that  they  would  in 
the  real  battlespace  -  via  verbal  interactions  using  specific 
communication  networks  and  through  digital  data  links. 
In  general,  however,  the  off-platfonn  entities 
communicate  with  the  E-2C  only  occasionally,  and  do  so 
in  a  highly  stylized  and  structured  means  relying  on  well- 
defined  syntax,  phraseology,  and  content.  Their  main 
focus  is  on  piloting  their  vehicles  and  completing  their 
portion  of  the  air  tasking  order.  These  behaviors  were 
already  largely  available  in  the  form  of  intelligent  pilots 
created  through  the  Soar  artificial  intelligence  technology. 
The  TacAir  Soar  pilots  [8]  were  already  compatible  with 
the  JSAF  battlespace,  and  had  some  ability  to 
communicate  verbally  with  standard  command  and 
control  entities  [11].  Thus,  the  TacAir  Soar  pilots  could 
be  integrated  to  provide  the  synthetic  off-platform 
teammates,  with  a  modest  effort  to  expand  their  verbal 
capabilities  to  include  the  structured  interactions  required 
with  the  E-2C. 

E-2C  Combat  System 


Figure  2.  SCOTT  Synthetic  and  Human  Entities 


The  on-platform  synthetic  teammates,  however,  required  a 
deeper  and  more  focused  understanding  of  the  command 
and  control  domain,  and  a  richer  means  of  communicating 
within  the  E-2C  tactical  team.  These  on-platform 
teammates  communicate  more  frequently  with  the  ACO, 
using  a  variety  of  communication  strategies  ranging  from 
spontaneous  speech  to  structured  communications,  and 


including  both  direct  communication  and 
understanding/processing  of  overhead  communications. 
In  addition,  the  on-platform  synthetic  teammates  had  to 
interact  with  the  E-2C  crewstations  in  a  functional 
manner.  Finally,  the  on-platform  synthers  offered  an 
instructional  opportunity,  by  enabling  an  even  greater 
level  of  control  over  the  nature  and  delivery  of  the 
training  component  of  the  underlying  exercises.  For 
example,  the  ability  the  synthetic  teammates  could  be 
manipulated  to  induce  errors  and  thus  provide  a  training 
opportunity  for  the  human  trainee  (e.g.,  provide  backup 
and/or  compensatory  behaviors).  For  such  reasons,  the 
on-platform  synthetic  teammates  are  implemented  as  fully 
embodied  cognitive  models  (cf.,  [12]).  Such  cognitive- 
model  based  synthetic  interactive  teammates  have  been 
termed  “synthers’  [5].  The  underling  technology  used  to 
create  the  SCOTT  synthers  is  the  CGF-COGNET  system 
and  iGEN™  software. 

The  overall  internal  architecture  of  CGF-COGNET-based 
synthers  is  pictured  in  Figure  3.  Additional  details  on 
CGF-COGNET  can  be  found  elsewhere  in  this  volume 
(see  [13]).  The  architecture  pictured  in  Figure  3  is  turned 
into  a  synther  by  defining  and  representing  the  knowledge 
needed  by  each  component  of  the  cognitive  system: 

•  perceptual  knowledge,  used  by  the  perceptual  process 
interprets  visual,  auditory,  and  linguistic  cues  and 
internalizes,  in  symbolic  form,  the  information  they 
contain, 

•  declarative  knowledge,  maintained  in  memory,  which 
represents  the  categorical,  indicative,  and  associational 
structure  of  the  domain, 

•  procedural  knowledge,  used  by  the  cognitive  process 
to  activate  domain-based  goals  (through  a 
recognitional  process)  and  accomplish  them  through 
application  of  domain  knowledge  and  use  of  physical 
instrumentalities  (e.g.,  manipidating  the  crewstation, 
speaking  to  teammates), 

•  metacognitive  knowledge,  which  defines  the  contexts 
by  which  various  goals  are  activated,  the  situational 
priorities  by  which  they  compete  for  attention,  and  the 
self-awareness  by  which  the  problem-solving  process 
is  managed  to  account  for  factors  such  as  workload, 
physical  constraints  (i.e.,  ears  can’t  listen  to  three 
messages  simultaneously),  and  interruptions;  and 

•  motor  knowledge,  used  by  the  motor  system,  to 
translate  intentions  for  action  into  specific  physical 
manipulations  of  the  outside  world 


The  Outside  World  (External  context) 


Visual  &  auditory  cues  Physical  &  verbal  actions 


Figure  3.  SYNTHER  Cognitive  Architecture 


4.1  Domain  Knowledge  and  Reasoning 

All  the  on-platform  synthers  for  SCOTT  are  built  from  a 
common  empirical  analysis,  largely  produced  by  the  Fleet 
Integration  Training  and  Evaluation  Research  (FITER) 
Program  [14].  FITER  performed  detailed  cognitive  task 
analyses  that  focused  on  team  shared  knowledge,  task 
strategies,  and  team  skills  used  by  a  naval  air  wing  at  the 
Fallon  Naval  Air  Station. 

Several  aspects  of  domain  knowledge  are  common  to  the 
E-2C  tactical  team.  First,  they  must  mentally  build  and 
maintain  a  spatial  and  a  temporal  representation  of  radar 
tracks  as  viewed  through  their  respective  display  consoles. 
Second,  they  must  possess  an  understanding  of  the 
capabilities  and  limitations  of  weapons  platforms  in  terms 
of  weapon  release  ranges,  electromagnetic  signal  (ES) 
signatures,  as  well  as  radar  coverage  zones  and  how  they 
relate  to  the  dynamic  tactical  situation  in  terms  of 
identification  disambiguation.  Third,  they  must  maintain 
constant  communication  with  the  elements  of  the  strike 
package  and  work  with  them  to  maintain  pre-planned 
schedules  and  tasking  while  responding  to  such  dynamic 
tactical  situations  as  pop-up  SAM  sites  and  enemy  aircraft 
launches.  A  number  of  additional  key  tactical  concepts 
from  FITER  analyses  are  incorporated  into  the  SCOTT 
CICO  synther  (Table  1). 


Table  1.  FITER  Tactical  Concepts 


Tactical  Concept 

Definition 

Situational 
awareness  (SA) 

“  a  large  scale  mental 

representation  of  the  tactical 
situation  held  by  aircrews. . .” 

Allowable  risk 

“...aircrews  draw  on  SA  to 
evaluate  whether  to  prosecute  an 
engagement,  weighing  mission 
objectives,  rules  of  engagement, 
and  survivability” 

Flow 

“...a  direction  of  movement  or 
apparent  movement. .  .Both  friendly 
and  threat  aircraft  have  flow,  and 
each  influences  the 

other. . .  experienced  aircrews 

establish  a  flow  that  will  allow 
them  to  achieve  the  mission’s 
objectives” 

Deconfliction 

“...the  design  and  maintenance  of 
flow  and  direction  of  travel  to 
minimize  the  risk  of  potential 
fratricide” 

Sanctuary 

“...the  creation  of  a  haven  in  time 
or  space  during  the  mission  to 
allow  an  element  to  conduct  an 
action  without  restriction” 

The  SCOTT  synther  must  be  able  to  represent  the 
essential  task  work  of  a  CICO,  a  great  deal  of  which  is 
either  directly  interacting  with  on-  and  off-platform  team 
members  or  monitoring  other  team  member  interactions 
in  order  to  maintain  overall  situational  awareness  and 
ensure  mission  success.  In  doing  this,  it  combines  the 
background  knowledge  and  reasoning  capabilities 
(summarized  in  Table  1)  with  more  specific  procedural 
knowledge  that  allows  it  to: 

•  interact  with  teammates  by  recognizing  their  utterances 
and  responding  verbally, 

•  perform  the  requisite  task-level  work  of  the  CICO 
such  as  track  ID  assessment  and  ES  correlation, 

•  perform  critical  team-level  tasks  such  as  performing 
compensatory  behaviors  and  proactive  information 
exchange,  and 

•  ‘tutor'  the  ACO  trainee,  providing  selective  diagnosis, 
instructional  feedback,  and  remediation  while 
referencing  internal  situational  and  tactical 
understanding. 

This  last  capability  is  that  of  an  instructional  agent,  above 
and  beyond  its  role  as  synthetic  teammate. 

One  of  the  most  important  capabilities  of  the  CICO 
synther  is  the  ability  to  engage  in  verbal  interactions  with 
its  human  team  members  -  in  this  case,  the  ACO  trainee. 
The  interaction  is  in  two  forms:  “hearing”  the  ACO 
trainee  through  the  use  of  speech  recognition  subsystems 
and  “speaking”  to  the  ACO  trainee  through  the  use  of  a 
text-to-speech  subsystem.  In  this  manner,  the  CICO 


synther  is  able  to  interact  with  the  human  trainees  and 
communicate  essential  tactical  information  throughout  the 
entire  battlespace,  as  well  as  overhear  equally  important 
tactical  communications. 

Another  important  capability  of  the  CICO  synther  is  the 
ability  to  offer  real-time  verbal  feedback  to  the  ACO 
trainee.  The  feedback  is  based  on  departures  from 
expectations  generated  by  the  CICO  as  part  of  its 
situational  awareness  about  ACO-specific  actions  and  task 
requirements.  In  the  E2-C  context,  the  CICO  is 
responsible  for  overall  strike  mission  perfonnance  and, 
therefore,  needs  to  maintain  SA  at  multiple  levels  of 
abstraction  and  from  the  perspective  of  various  team 
members  internal  and  external  to  the  E-2C  platform.  It  is 
precisely  because  of  the  central  role  that  the  CICO  plays 
in  the  E2-C  that  much  of  the  instructional  context  for 
monitoring  the  ACO  trainee  has  been  derived  from  the 
perspective  of  the  CICO. 

5.  Conclusions 

SCOTT  is  being  implemented  and  demonstrated  at  the 
Synthetic  Forces  laboratory  at  the  Naval  Air  Warfare 
Center  Training  System  Division,  Orlando,  FL.  In 
addition  to  the  E-2C  SCOTT,  an  inter-domain  validation 
of  the  SCOTT  architecture  is  being  undertaken  by 
applying  the  same  approach  to  the  operating  environment 
of  the  Virtual  Environment  Landing  Craft  Air  Cushion 
(VE-LCAC).  The  goal  of  the  VE-LCAC  effort  is  to 
develop  a  prototype  training  system  capability  that  allows 
any  LCAC  crewmember  (craft  master,  navigator,  or 
engineer)  to  train  in  real  time  with  synthetic 
crewmembers.  The  planned  extension  of  the  SCOTT 
architecture  in  the  VE-LCAC  domain  will  culminate  with 
a  human-in-the-loop  demonstration  of  the 
testbed/prototype  training  system  in  which  a  simulated 
crewmember  participates  in  the  execution  of  an 
amphibious  assault  mission  scenario.  Extension  of  the 
SCOTT  architecture  into  a  virtual  testbed  environment 
will  confirm  the  robustness  of  the  approach  in  meeting 
training  requirements  and  supporting  optimized 
manpower  in  different  development  frameworks  and 
training  application  areas. 
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